Developing Toward an SLA: Establishing a Test Plan

The previous installments of this series discussed several facets of evaluating a Service Level Agreement (SLA). In [Wil10b], we examined the variation of the response times across transaction types. This was a notional analysis since the data were designed specifically to illustrate the points. In [Wil10a], we looked at the variation in average response time due to the load. The different types of load correspond to different parts of the SLA evaluation period. We also took a real transaction and analyzed its response time distribution. In [Wil11a], we considered how transaction performance varies due to the data referenced by the transaction. We highlighted that the results for the performance test are likely to be influenced by the data chosen for the test. This aspect is complicated by the growth of the user base and the database over the operations lifecycle. In [Wil11b], we considered how user load varies during the SLA evaluation period and the operations lifecycle. We observed that there was a load difference between a mixed system and a closed one. In this installment, we are going to bring all of these concepts together into a plan that can control both the risk of system performance and its cost. The plan has options that the responsible decision-maker will have to consider in conjunction with the myriad of other system development and maintenance issues. Work avoided in the development phase increases the dangers associated with poor design in the maintenance (or operations) phase. This is a conscious choice.